1. Field of Invention
The present invention relates generally to manipulation of electronic media using off-line media. More specifically, the present invention is related to synchronizing the actions of a user manipulating off-line media to responses in electronic media.
2. Discussion of Related Art
Children and students benefit from having a variety of tools (or instruments) to study the world including microscopes, telescopes, etc. These tools allow the individuals to explore the world through direct manipulation and observation. Often, rather than having availability of such tools, the children or students learn about the information these tools provide through static images presented to them in books or displayed on a computer screen. There is a context and visceral experience of moving a tool that is lost when the images from these tools are presented in such a static manner. Some tools, no matter what the availability, are beyond the means of children and students to use because of factors like size, cost, complexity and safety. Examples include atomic force microscopes, mass spectrometers, magnetic resonance imagers, Geiger counters and electrophoretic devices.
Simulation machines have often been used in some fields to provide the same visceral experience to a user in similar situations, i.e., when the expense is high or availability of equipment is low. For instance, flight simulators have often been used to train pilots without actually having to provide a plane. In addition, medical operations and diagnosis have been simulated in order to give students or professionals experience, without actually having to practice on a patient. Normally, simulators are used to give true to life experience, without the expense associated with a true experience. Most simulators themselves are expensive pieces of equipment and not widely available. The following provide some examples of simulation systems.
The U.S. Pat. No. 5,609,485 to Bergman et al. describes a medical reproduction system, which is designed to train medical physicians and technicians in the use of ultrasound equipment. Biological data is collected from a living human and stored in memory. A user then manipulates a transmitter over a simulated body. The transmitter sends position data to a receiver and, based upon the position, corresponding, preset biological data is displayed.
The U.S. Pat. No. 5,766,016 to Sinclair et al. describes a surgical simulator, which provides visual and tactile feedback to portray an actual surgical procedure. Feedback is generated by an SGI computer to display two images at slightly different perspectives, which are combined to form a stereoscopic image. Force feedback motors are provided to generate resistive force to an instrument utilized to simulate a surgical tool.
The U.S. Pat. No. 5,800,178 to Gillio describes a virtual surgery system, which includes a box having entrances thereon. Simulated tools are inserted through the entrances and image data of a patient is utilized to provide the virtual surgery. Force feedback is provided with the images to provide resistive forces to the simulated tools.
The above examples are expensive systems, and due to the specialty of their purpose are not available to everyone generally. These devices do not provide people in general with a manner to explore aspects of the world normally unavailable to them in a manner that is the most productive, i.e., with a visceral experience.
There have also been efforts to provide people with a more interactive experience with static printed materials, such as books. The U.S. Pat. Nos. 5,749,735 and 5,839,905 to Redford et al. describe an interactive system in which a printed publication has a button or buttons associated therewith. When a user activates the buttons, it remotely controls a display to display content associated with the button. This system does not truly provide a visceral experience as there is no direct manipulation of a hand-held device, merely the pressing of a button, which calls up associated content.
Also, people often prefer reading and manipulating off-line media (e.g. paper, magazines, newspapers and books) than electronic media (computers displaying on CRT, mobile phone and personal digital assistants displaying on small LCD screens). Off-line media typically has better resolution, contrast and viewing area, is lighter weight, less expensive, and portable than digital media. For example when writing a large paper, people tend to print out a copy and edit with pencil, then enter the changes into their computer. Many people read books and underline important passages, words or references. Cross word puzzles appear in newspapers and magazines are filled out with pencil or pen. Bookmarks or paper clips are placed in books to mark pages. Magazines featuring full picture advertisements and mail-order catalogs provide a better user experience than electronic catalogs due to their ease-of-use and high quality presentation. People cut out or circle articles and pictures from magazines and newspapers with ease. However, digital electronic media offers better searching, indexing, manipulation, distribution, and communication than off-line media can provide. For example calling in an order by phone is a slow process and requires a human operator. Filling out a form and mailing it is even a slower process. Yet finding the correct URL for a web page can be difficult, as URLs are not necessarily the exact company name, and further, a company may have hundreds or thousands of pages, making finding a particular page difficult.
Therefore, it is desired to provide a means for people to manipulate familiar and comfortable off-line media, and simultaneously effect electronic media to take advantage of the searching, indexing, referencing, and manipulation capabilities of digital electronic media.
There have been efforts to allow customers to scan bar codes printed on off-line material, and use the bar code to provide digital links to products. The CueCat ® from Digital Convergence is a bar code reader and software. When the bar code reader is swiped across a product with a UPC (Universal Product Code), the product's web page appears on the computer.
Anoto Corporation provides a pen that reads tiny dot codes printed in newspaper advertisements. When a user sees an ad for flowers and fills out the form, the pen wirelessly transmits the handwriting and user id to the user's mobile phone, and their order is placed electronically.
However, both these systems of bar codes and dot codes require specially prepared off-line media and do not allow manipulation of the electronic media. Advertisers are reluctant to add special bar codes or dot codes to their advertisements in newspapers and magazines without a proven benefit. Yet it is difficult to prove a benefit with prior art if the special bar or dot codes are not inserted in the advertisements. It would be advantageous to provide a means to allow synchronization of electronic media with unmodified off-line media. For example, a company could provide synchronization to magazines and newspapers without modifying the images and text printed in the magazines and newspapers.
The U.S. Pat. No. 5,420,943 to Mak describes an input device in which a CCD device is utilized to scan an off-screen medium in order to effect input. One embodiment of Mak utilizes unique positions on the off-screen medium, which are utilized to input ASCII characters or commands. However, Mak does not synchronize the unique position with corresponding multimedia data to be displayed by the computing device.
The U.S. Pat. No. 5,601,435 describes a system in which physiological monitors attached to an individual are used as inputs to specialized software which leads the user through a medical diagnosis and further provides advice about whether to see a caregiver, diet, health habits, etc.
The U.S. Pat. No. 5,825,009 to Schmid et al. and U.S. Pat. No. 5,854,945 to Criscito et al. describe systems for inputting bar code data through a keyboard port or buffer of a computer.
Whatever the precise merits, features and advantages of the above cited references, none of them achieve or fulfills the purposes of the present invention.